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' The randomness rate of an infinite binary sequence is characterized by the sequence of 

D , ratios between the Kolmogorov complexity and the length of the initial segments of the 

' sequence. It is known that there is no uniform effective procedure that transforms one input 

sequence into another sequence with higher randomness rate. By contrast, we display such 
a uniform effective procedure having as input two independent sequences with positive but 
arbitrarily small constant randomness rate. Moreover the transformation is a truth-table 
reduction and the output has randomness rate arbitrarily close to 1. 
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1 Introduction 

> 

00 ' It is a basic fact that no function can increase the amount of randomness (i.e., entropy) of 

■ a finite structure. Formally, if X is a distribution on a finite set A, then for any function / 

mapping A into A, the (Shannon) entropy of f{X) cannot be larger than the entropy of X. As 
^ it is usually the case, the above fact has an analogue in algorithmic information theory: for any 

' finite binary string x and any computable function /, K[f{x)) < K{x) + 0(1), where K{x) is 

. the Kolmogorov complexity of x and the constant depends only on the underlying universal 

machine. The above inequality has an immediate one-line proof, but the analoguous statement 
when we move to infinite sequences is not known to hold. For any a G [0, 1], we say that an 
^ . infinite binary sequence x has randomness rate a, if i^(x(l : n)) > an for all sufficiently large 

^ \ n, where x{l : n) denotes the initial segment of x of length n0 The question becomes: if 

x has randomness rate < cr < 1, is there an effective transformation / such that f{x) has 
randomness rate greater than that of x? Unlike the case of finite strings, infinite sequences 
with positive randomness rate possess an infinite amount of randomness (even though it is 
sparsely distributed) and thus it cannot be ruled out that there may be a way to concentrate 
it and obtain a sequence with higher randomness rate. 

This is a natural question, first raised by Reimann |Rei04j . which has received significant 
attention recently (it is Question 10.1 in the list of open questions of Miller and Nies [MN06| ). 
So far, there exist several partial results, mostly negative, obtained by restricting the type of 
transformation. Reimann and Terwijn |Rei04t Th 3.10] have shown that for every constant 
c < 1, there exists a sequence x such that if / is a many-one reduction, then the randomness rate 
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'^The randomness rate of x is very close to the notion of constructive Hausdorff dimension of x |Lut03l[May02| 
|Rya84[ [ Sta05) : however since this paper is about handling randomness and not about measure-theoretical issues 
we prefer the randomness terminology. 
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of f{x) cannot be larger than c. This result has been improved by Nies and Reimann |NR06] 
to wtt-reductions. More precisely, they showed that for all rational c G (0,1), there exists a 
sequence x with randomness rate c such that for all wtt-reductions /, /(x) has randomness 
rate < c. Bienvenu, Doty, and Stephan |BDS07j have obtained an impossibility result for 
the general case of Turing reductions, which, however, is valid only for uniform reductions. 
Building on the result of Nies and Reimann, they show that for every Turing reduction / and 
all constants ci and C2, with < ci < C2 < 1, there exists x with randomness rate > ci such 
that f{x), if it exists, has randomness rate < C2. In other words, loosely speaking, no effective 
uniform transformation is able to raise the randomness rate from ci to C2. Thus the question 
"Is there any effective transformation that on input a € (0,1], e > 0, and x, a sequence 
with randomness rate a, produces a string y with randomness rate cr + e ?" has a negative 
answer. On the positive side. Doty jDot07j has shown that for every constant c there exists 
a uniform effective transformation / able to transform any x with randomness rate c € (0, 1] 
into a sequence f{x) that, for infinitely many n, has the initial segments of length n with 
Kolmogorov complexity > (1 — e)n (see Doty's paper for the exact statement). However, since 
Doty's transformation / is a wtt-reduction, it follows from Nies and Reimann's result that 
f{x) also has infinitely many initial segments with no increase in the Kolmogorov complexity. 

In the case of finite strings, as we have observed earlier, there is no effective transformation 
that increases the absolute amount of Kolmogorov complexity. However, some positive results 
do exist. Buhrman, Fortnow, Newman, and Vereshchagin [BFNV05] show that, for any non- 
random string of length n, one can flip 0{^/n) of its bits and obtain a string with higher 
Kolmogorov complexity. Fortnow, Hitchcock, Pavan, Vinodchandran, and Wang [FHP"'"06] 
show that for any < a < f3 < 1, there is a polynomial-time procedure that on input x with 
K(x) > a\x\, using a constant number of advice bits (which depend on x), builds a string y 
with K{y) > (3\y\ and y is shorter than x by only a multiplicative constant. 

Our main result concerns infinite sequences and is a positive one. Recall that Bienvenu, 
Doty and Stephan have shown that there is no uniform effective way to increase the randomness 
rate when the input consists of one sequence with positive randomness rate. We show that 
if instead the input consists of two such sequences that are independent, then such a uniform 
effective transformation exists. 

Theorem 1.1 (Main Result) There exists an effective transformation f : Q x {0,1}°° x 
{0, 1}°° {0, 1}°° with the following property: If the input is t ^ (0, 1] and two indepen- 
dent sequences x and y with randomness rate r, then f{T,x,y) has randomness rate 1 — S, for 
all 6 > 0. Moreover, the effective transformation is a truth-table reduction. 

Effective transformations are essentially Turing reductions that are uniform in the parameter r; 
see Section[2j Two sequences are independent if they do not contain much common information; 
see Section [3l 

One key element of the proof is inspired from Fortnow et al.'s [FHP"'"06] . who showed that 
a randomness extractor can be used to construct a procedure that increases the Kolmogorov 
complexity of finite strings. Their procedure for increasing the Kolmogorov complexity runs in 
polynomial time, but uses a small amount of advice. To obtain the polynomial-time efficiency, 
they had to use the multi-source extractor of Barak, Impagliazzo, and Wigderson |BIW04] . 
which requires a number of sources that is dependent on the initial min-entropy of the sources 
and on the desired quality of the output. In our case, we are not concerned about the efficiency 
of the transformation (this of course simplifies our task), but, on the other hand, we want it 
completely effective (with no advice), we want it to work with just two sources, and we want 
it to handle infinite sequences. In place of an extractor, we provide a procedure with similar 
functionality, using the probabilistic method which is next derandomized in the trivial way 
by brute force searching. Since we handle infinite sequences, we have to iterate the procedure 



2 



infinitely many times on finite blocks of the two sources and this necessitates solving some 
technical issues related to the independence of the blocks. 

2 Preliminaries 

We work over the binary alphabet {0, 1}. A string is an element of {0, 1}* and a sequence is 
an element of {0, 1}°° . If x is a string, |x| denotes its length. If x is a string or a sequence 
and n,ni,n2 £ N, x{n) denotes the re-th bit of x and x{ni : 712) is the substring x(ni)x(ni + 
1) . . .x(n2). The cardinality of a finite set A is denoted ||A||. Let M be a standard Turing 
machine. For any string x, define the (plain) Kolmogorov complexity of x with respect to M, 
as 

Km{x) = min{|p| | M{p) = x}. 

There is a universal Turing machine U such that for every machine M there is a constant c 
such that for all x, 

Ku{x)<Km{x)+c. (1) 

We fix such a universal machine U and dropping the subscript, we let K{x) denote the Kol- 
mogorov complexity of x with respect to U . For the concept of conditional Komogorov com- 
plexity, the underlying machine is a Turing machine that in addition to the read/work tape 
which in the initial state contains the input p, has a second tape containing initially a string 
y, which is called the conditioning information. Given such a machine M, we define the 
Kolmogorov complexity of x conditioned by y with respect to M as 

Km{x I y) = min{|p| | M{p,y) = x}. 

Similarly to the above, there exist universal machines of this type and they satisfy the relation 
similar to Equation [H but for conditional complexity. We fix such a universal machine U, and 
dropping the subscript U, we let K{x \ y) denote the Kolmogorov complexity of x conditioned 
by y with respect to U. 

We briefly use the concept of prefix-free complexity, which is deflned similarly to plain 
Kolmogorov complexity, the difference being that in the case of prefix-free complexity the 
domain of the underlying machines is required to be a prefix-free set. 

Let cj G [0, 1]. A sequence x has randomness rate a if K(x{l : n)) > cr ■ n, for almost every 
n (i.e., the set of n's violating the inequality is finite). 

An effective transformation / is represented by a two-oracle Turing machine Mf. The 
machine Mf has access to two oracles x and y, which are binary sequences. When Mj makes 
the query "n-th bit of first oracle?" ("n-th bit of second oracle?"), the machine obtains x{n) 
(respectively, y{n)). On input (r, 1"), where r is a rational (given in some canonical represen- 
tation), Mf outputs one bit. We say that f{T,x,y) = z £ {0,1}°°, if for all n, Mf on input 
(r, 1") and working with oracles x and y halts and outputs z{n). (Effective transformations are 
more commonly called Turing reductions. If r would be embedded in the machine Mf, instead 
of being an input, we would say that z is Turing-reducible to (x, y). Our approach emphasizes 
the fact that we want a family of Turing reductions that is uniform in the parameter r.) In 
case the machine Mf halts on all inputs and with all oracles, we say that / is a truth-table 
reduction. 

3 Independence 

We need to require that the two inputs x and y that appear in the main result are really 
distinct, or in the algorithmic-information theoretical terminology, independent. 
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Definition 3.1 Two infinite binary sequences x,y are independent if for all natural numbers 
n and m, 

K{x{l : n)y{l : m)) > K{x{l : n)) + K{y{l : m)) - 0(log(n) + log(m)). 

The definition says tliat, modulo additive logarithmic terms, there is no shorter way to 
describe the concatenation of any two initial segments of x and y than having the information 
that describes the initial segments. 

It can be shown that the fact that x and y are independent is equivalent to saying that for 
every natural numbers n and m, 

K{x{l : n) I y(l : m)) > K{x{l : n)) - 0(log(n) + log(m)). (2) 

and 

K{y{l : m) \ x{l : n)) > K{y{l : m)) - 0(log(n) + log(m)). (3) 

Thus, if two sequences x and y are independent, no initial segment of one of the sequence 
can help in getting a shorter description of any initial segment of the other sequence, modulo 
additive logarithmical terms. 

In our main result, the input consists of two sequences x and y that are independent and 
that have Kolmogorov rate a for some positive constant a < 1. We sketch an argument showing 
that such sequences exist. In our sketch we take a = 1/2. 

We start with an arbitrary random (in the Martin-L6f sense) sequence x. Next using 
the machinery of Martin-Lof tests relativized with x we infer the existence of a sequence y 
that is random relative to x. From the theory of Martin-Lof tests, we deduce that there 
exists a constant c such that for all m, H{y{l : m) \ x) > m — c, where H{-) is the prefix- 
free version of complexity. Since H{y{l : m)) < m + O(logm), for all m, we conclude that 
H{y{\ : m) \ x) > H{y{l : m)) — O(logm), for all m. Therefore, H{y[l : m)) \ x{\ : n)) > 
H{y{\ : m) \ x) — O(logn) > H[y{\ : m)) — 0(logn + logm), for all n and m. Since the 
prefix-free complexity H{-) and the plain complexity K{-) are within O(logm) of each other, 
it follows that K{y{l : m)) \ x(l : n)) > K[y{l : m)) — 0(logn + logm)), for all n and m. This 
implies K{x{l : m)y{l : n)) > K{x{l : n)) + K{y{l : m)) — 0(log(n) + log(m)), for all n, m. 
Next we construct x' and y' by inserting in x and respectively y, the bit in all even positions, 
i.e., x' = 2;iOx20 . . . (where Xi is the i-th bit of x) and y' = yi0y20 ■ ■ ■■ Clearly, K{x{l : n)) and 
K(xiO . . . XnO) are within a constant of each other, and the same holds for y and y' . It follows 
that x' and y' are independent and have randomness rate 1/2. 

4 Proof of Main Result 
4.1 Proof Overview 

We present in a simplified setting the main ideas of the construction. Suppose we have two 
independent strings x and y of length n such that K(x) = an and K{y) = an, for some o" > 0. 
We want to construct a string z of length m such that K(z) > (1 — e)m. The key idea (borrowed 
from the theory of randomness extractors) is to use a function E : {0, 1}" x {0, 1}" {0, l}*" 
such that every large enough rectangle of {0, 1}" x {0, 1}" maps about the same number of pairs 
into all elements of {0, 1}*". We say that such a function is regular (the formal Definition [4]8] has 
some parameters which quantify the degree of regularity). To illustrate the idea, suppose for a 
moment that we have a function E : {0, 1}" x {0, 1}" {0, 1}™ that, for all subsets B C {0, l}"" 
with ||i?|| ~ 2^^", has the property that any a G {0, 1}™' has the same number of preimages in 
BxB, which is of course \\B x B||/2"". Then for any A C {0, 1}"", E-'^{A) n{Bx B) has size 
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• ll^ll- Let us take z = E{x,y) and let us suppose that K{z) < (1 — e)m. Note that the 
set B = {ue {0, 1}" I K{u) = an] has size ^ 2'^", the set A = {v e {0, 1}™ | < (l-e)m} 

has size < 2^^"'^^"^ and that x and y are in i?~^(j4) ri{B x B). By the above observation the set 
E-'^{A) r]{B xB) has size < ^"^5" • Since ^^-^(yl) D {B x B) can be enumerated effectively, 
any pair of strings in E~^(A) H B x B can be described by its rank in a fixed enumeration of 
E~^{A) n B X B. In particular {x,y) is such a pair and therefore K{xy) < 2(7n — em. On the 
other hand, since x and y are independent, K[xy) ~ -^^(a;) + -^(y) = 2o"?7-. The contradiction 
we have reached shows that in fact K{z) > (1 — e)m. 

A function E having the strong regularity requirement stated above may not exist. Fortu- 
nately, using the probabilistic method, it can be shown (see Section that, for all m < nP'^^'^, 
there exist a function E : {0, 1}" x {0, l}*^ {0, 1}'" such that all strings a G {0, 1}™" have at 
most 2{\\B X 511/2™) preimages in any B x B as above (instead of {\\B x B\\/2''^) preimages 
in the ideal, but not realizable, setting we used above). Once we know that it exists, such 
a function E can be found effectively by exhaustive search. Then the argument above, with 
some minor modifications, goes through. In fact, when we apply this idea, we only know that 
K{x) > an and K{y) > an and therefore we need the function E to satisfy a stronger variant 
of regularity. However, the main idea remains the same. 

Thus there is an effective way to produce a string z with Kolmogorov complexity (1 — e)m 
from two independent strings x and y of length n and with Kolmogorov complexity an. Recall 
that, in fact, the input consists of two independent infinite sequences x and y with randomness 
rate r > 0. To take advantage of the procedure sketched above which works for finite strings, 
we split X and y into finite strings xi, X2, • • • , . . ., and respectively yi,y2, . . . ,yn, ■ ■ ■, such 
that the blocks Xi and yi, of length nj, have still enough Kolmogorov complexity, say (r/2)?ij, 
conditioned by the previous blocks xi, . . . , Xi-i and yi, . . . , yi-i. The splitting of x and y into 
blocks and the properties of the blocks are presented in Section 14.21 Then using a regular 
function Ei : {0, 1}"' x {0, 1}"' {0, 1}™' , we build Zi = Ei{xi, yi). By modifying shghtly the 
argument described above, it can be shown that K{zi \ xi, . . . , yi, . . . , yi-i) > (1 — e)mj, 
i.e., Zi has high Kolmogorov complexity even conditioned by the previous blocks xi, . . . ,Xi-i 
and It follows that K{zi \ zi,...,Zi-i) is also close to mj. We finally take 

z = ziZ2 . . ., and using the above property of each Zi, we infer that for every n, the prefix of z 
of length n has randomness rate > (1 — e)n. In other words, z has randomness rate (1 — e), as 
desired. 

4.2 Splitting the two inputs 

The two input sequences x and y from Theorem ll.ll are broken into finite blocks xi,X2, . . . ,Xi, . . . 
and respectively yi,y2, . . . ,yi, . . .. The division is done in such a manner that Xi (respectively, 
yi) has high Komogorov complexity rate conditioned by the previous blocks xi, . . . ,Xi-i (re- 
spectively by the blocks yi, . . . , yi-i). The following lemma shows how this division is done. 

Lemma 4.1 (Splitting lemma) Let x G {0,1}°° with randomness rate r, for some constant 
T > 0. Let < a < T. For any no sufficiently large, there is ni > no such that 

K{x{no + 1 : ni) | x(l : no)) > a{ni — no). 

Furthermore, there is an effective procedure that on input no, r and a calculates ni. 

Proof. Let a' be such that < a' < t — a. Take 

pl - fj. 
ni = Po- 
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Suppose K{x{nQ + 1 : ni) | x(l : no)) < (T(ni — no). Then x(l : ni) can be reconstructed from: 
x(l : no), the description of x(no + 1 : ni) given x{l : no), no, extra constant number of bits 
describing the procedure. So 



Now we define the points where we spht x and y, the two sources. 

Take a, the point from where the Sphtting Lemma holds. For the rest of this section we 



The fohowing sequence represents the cutting points that wih define the blocks. It is defined 
recursively, as follows: to = 0, ti = a, U = h{ti + . . .+tj_i). It can be seen that ti = a6(l+6)*~^, 
for i > 2. 

Finally, we define the blocks: for each i > 1, := x{ti-i + 1 : tj) and yi = y{ti-i + 1 : tj), 
and Tii := |xj| = \yi\ = ah^{l + 6)*~^ (the last equality holds for i > 3). 

We also denote by Xi the concatenation of the blocks xi, . . . , Xj and by yi the concatenation 
of the blocks yi , . . . , yj . 

Lemma 4.2 1. K{xi \ Xi-i) > arii, for all i > 2 (and the analogue relation holds for the 
Vi 's). 

2. log |xj| = Q{{) and log |xj| = 0(i), for all i (and the analogue relation holds for the yi 's). 

Proof. The first point follows from the Splitting Lemma 14.11 and the second point follows 
immediately from the definition of n^ (which is the length of Xj) and of ti (which is the length 
ofxi). I 

The following facts state some basic algorithmic-information theoretical properties of the 
blocks xi, X2, .... and yi, y2, .... 

We first recall the following basic fact (for example, see Alexander Shen's lecture notes [SheOOj ). 

Theorem 4.3 For all finite binary strings u and v, 

(i) K{vu) < K{u)+K{v I u) + O {log K{u)+ log K{v)). 

(ii) K{vu) > K{u)+K{v I u) - O {log K{u)+ log K{v)). 

The hidden constants depend only on the universal machine that defines the complexity K{-). 

Lemma 4.4 For all finite binary strings u and v, 

\K{v I u) - {K{vu) - K{u)) I < 0(log \u\ + log \v\). 

Proof. Theorem 14.31 implies \K{v \ u) — {K{vu) — K{u))\ < 0{logK{u) + logK{v)). Since 
K{u) < \u\ + 0(1) and K{v) < \v\ + 0(1), the conclusion follows. | 

Lemma 4.5 For all i and j, 



K{x{l : ni)) < no + a{ni - no) + log no + 0{l) 



= aui + (1 - fT)no + log no + 0(1) 

< aui + a'ui + log no + 0(1) 

< rni (if no suffic. large) , 



(4) 



which is a contradiction if m is sufficiently large. 



I 




(a) \K{y,x^) - {K{yi) + K{xj))\ < 0{i+j). 

(b) \K{xiy,) - {K{xi)+K{yj))\ < 0{i+j). 
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Proof. We prove (a) ((6) is similar). 

KiviXj) < Kivi) + K{xj) + 0{\og{K{y,) + \og{K{x,)) 

< Kivi) + Kixj) + 0(log \yi\ + log (5) 
= E:(yi)+/^(:E,)+0(z+i). 

The first line follows from Theorem 14.31 (i) (keeping in mind that K(v\u) < K{v) +0(1)). For 
the last line we took into account that log \xj\ = 0{j) and log \yi\ = 0{i). 
On the other hand, 



KiviXj) > K{yi) + K{xj) - 0(log \yi\ + log |%| 
= K{yi)+K{x.j)-0{i+j). 



(6) 



The first line follows from the independence of x and y. 

Combining equations ([5]) and ([H), the conclusion follows. | 

Lemma 4.6 For all i and j, 

(a) \K{xi I Xi_iyj) - K{xi \ < 0{i+ j). 

(h) \K{yi I Xjyi-i) - K{yi \ yi-i)\ < 0{i + j). 
Proof. We prove (a) ((6) is similar). We first evaluate K{xi \ Xi-iyj). From Lemma 14.41 

\K{xi I Xi-iyj) - {K{xiXi-iyj) - K{xi-iyj)) \ < 0{i + j). (7) 

It is easy to check that K{xiXi-iyj) is within 0{i) from K{xi-iXiyj). Thus, we can substitute 
K{xiXi-iyj) by K{xiyj) and obtain, 

\K{xi I Xi^iyj) - {K{xiyj) - K{xi^iyj)) \ < 0{i + j). (8) 

Next, by Lemma iSl \K{xiyj) - {K{xi) + K{yj)) \ < 0{i + j) and \K{xi-iyj) - {K{xi^i) + 
K{yj)^ \ < 0{i + j). Plugging these inequalities in Equation we get 

\K{xi I Xi-iyj) - {K{xi)-K{xi^i))\ < 0{i + j). (9) 

We next evaluate K{xi \ Xi-i). From Lemma |4.4|, 

\K{xi I Xi^i) - {K{xiXi.i) - K{xi^i)) \<0{i+ j). (10) 

Using the inequality \K{xiXi-i) — K{xi-iXi)\ < 0(1), we obtain 

\K{xi I Xi_i) - {K{x^) - K{x,^i)) \<0{i + j). (11) 

From Equations 1^ and (jlip . the conclusion follows. | 

Lemma 4.7 For all i, K{xiyi \ Xi_iyj_i) > K{xi \ xi-iyi-i) + K{yi \ Xi_iyj_i) - 0{i). 

Proof. The conditional version of the inequality in Theorem 14.31 holds true, i.e., for all strings 
u,v and K{uv \ w) > K{u \ w) + K{v \ uw) — 0{log K{u) + logK{v)). Thus, keeping into 
account that K{xi) < \xi\ + 0(1) = 2'^(*) and K{yi) < \yi\ + 0(1) = 2^^^\ we get 

K{xiyi I Xi-m^i) > K{xi I Xi_iyj_i) + K{yi \ XiXi-iyi^i) - 0{i). 

Note that K{yi \ XjXj_iyj_i) and K{yi \ Xiyi-i) are within a constant of each other, and 
therefore 

K{xiyi I Xi_iyi_i) > K{xi \ Xi^iy-i^i) + K{yi \ Xiyi^i) - 0{i). 

Next, we note that K{yi \ Xiyi^i) > K{yi \ i/j_i) - 0{i) > K{yi \ Xi-iyi-i) - 0{i), where the 
first inequality is derived from Lemma 14.61 The conclusion follows. I 
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4.3 Regular functions 

The construction of z from x and y proceeds block-wise: we take as inputs the blocks Xi and yj 
and, from them, we build Zj, the i-th block of z. The input strings Xj and yj, both of length n^, 
have Kolmogorov complexity trnj, for some positive constant cr, and the goal is to produce Zj, 
of length mi (which will be specified later), with Kolmogorov complexity (1 — e)mi, for positive 
e arbitrarily small. This resembles the functionality of randomness extractors and, indeed, the 
following definition captures a property similar to that of extractors that is sufficient for our 
purposes. 

Definition 4.8 A function f : {0, 1}" x {0, !}"■ {0, 1}*" is {a, c)-regular, if for any ki,k2 > 
an, any two subsets Bi C {0, 1}" and B2 C {0, 1}" with \\Bi\\ = and \\B2\\ = 2^'^ have the 
following property: for any a G {0, l}™, 

||ri(a)n (Six 52)11 <^||Six52||. 

Remarks: Let [N\ be the set {1, . . . , N}. We identify in the standard way {0, 1}" with [N], 
where N = 2^. We can view [N] x [A'^] as a table with N rows and N columns and a function 
/ : [N] X [N] [M] as an assignment of a color chosen from [M] to each cell of the table. 
The function / is (o", c)-regular if in any rectangle of size [K] x [K], with k > an, no color 
appears more than a fraction of c/M times. (The notion of regularity is interesting for small 
values of c because it says that in all rectangles, unless they are small, all the colors appear 
approximately the same number of times; note that if c = 1, then all the colors appear the 
same number of times.) 

We show using the probabilistic method that for any a > 0, {a, 2)-regular functions exist. 
Since the regularity property for a function / (given via its truth table) can be effectively 
tested, we can effectively construct (cr, 2)- regular functions by exhaustive search 

We take / : [N] x [N] — > [M], a random function. First we show that with positive 
probability such a function satisfies the definition of regularity for sets A and B having size 
2^^, where k is exactly \an]. Let's temporarily call this property the weak regularity property. 
We will show that in fact weak regularity implies the regularity property as defined above (i.e., 
the regularity should hold for all sets Bi and B2 of size 2^^^ and respectively 2^^^ ^ fo^ j^^ ^nd ^2 
greater or equal \an]). 

Lemma 4.9 For every a > 0, if M < ivO-^^""^ then it holds with probability > that f satisfies 
the {a, 2) - weak regularity property as defined above. 

Proof.. 

Fix Bi C [A''] with ||-Bi|| = A''"" (to keep the notation simple, we ignore truncation issues). 
Fix B2 C [N] with \\B2\\ = N" . 

Let ji £ Bi X B2 and j2 € [M] be fixed values. As discussed above, we view [N] x [A'"] as 
a table with A^ rows and A^ columns. Then Bi x B2 is a rectangle in the table, ji is a cell in 
the rectangle, and j2 is a color out of M possible colors. 

Clearly, Prob(/(ji)=i2) = l/M. 

By Chernoff bounds, 

/ / no. of j2-colored cells in Bi x B2 _ ]_\ ^ J_\ ^ ^-(i/m)-n--n--{i/3) 

By the union bound 

Prob( the above holds for some j2 in [M] ) < Me-^^/^^'^'''^'' ^^/^). (12) 
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The number of rectangles Bi x B2 is 

• < {ii^fy = e^^" • e^^-a-)!-^. (13) 

Note that if there is no rectangle Bi x B2 and j2 as above, then / satisfies the weaker 
(cj, 2)-regularity property. 

Therefore we need that the product of the right hand sides in equations (fT2]) and (fT3l ) is 
< 1. 

This is equivalent to 

(1/M) -N^"" -1/3- ln(M) > 27^ + 2iV^ • (1 - fr) IniV, 

which holds true for M < N^-^^" . | 
As promised, we show next that weak regularity implies regularity. 

Lemma 4.10 Let f : {0, 1}" x {0, 1}" {0, 1}™ such that for every Bi C {0, 1}" with 
\\Bi\\ = 2^, for every B2 C {0, 1}" with \\B2\\ = 2^ , and for every a G {0, l}™" it holds that 

\\f-\a) n {Bi X B2)\\ < p. 

Then for every ki > k and every k2 > k, for every B[ C {0,1}" with \\B[\\ = 2^'^, for every 
B'2 C {0, 1}" with \\B'2\\ = 2'=2, and for every a G {0, 1}™ it holds that 

\\f-Ha)n{B[xB'2)\\<p. 

Proof.. We partition B[ and B2 into subsets of size 2^^. So, B[ = Ai U A2 U . ■ ■ U Ag, with 
pill = 2*^, i = 1,... ,s and ^2 = Ci U C2U . .. UCj, with ||Cj|| = 2*-', j = l,...,t. Then, 

||/-i(a) n iB[ X i?^)|| = n=i EU\\f-\a) n {A, X C,)\\ 

= p-||i?lxi?^||. 

4.4 Increasing the randomness rate 

We proceed to the proof of our main result, Theorem ll.il 

We give a "global" description of the effective mapping / : Q x {0, 1}°° x {0, 1}°° {0, 1}°°. 
It will be clear how to obtain the n-th bit of the output in finitely many steps, as it is formally 
required. 

Construction 

Input: T G (5 n (0, 1], x,y G {0,1}°° (the sequences x and y are oracles to which the 
procedure has access) . 

Step 1: Split x into xi, X2, • • • , a^j, • • • and split y into yi,y2, ■ ■ ■ ,yi, ■ ■ ■, as described in 
Section [4.21 taking a = t/2 and a' = r/4. 

For each i, let |xj| = \yi\ = ui (as described in Section HT2|) . 
By Lemma 1121 K{xi \ Xi-i) > ani and K^iyi \ Vi-i) > crui. 

Step 2: As discussed in Section [43| for each i, construct by exhaustive search Ei : {0, 1}"' x 
{0, 1}"' {0, 1}™' a (cr/2, 2)-regular function, where rrii = i'^. 

We recall that this means that for all ki, k2 > (cr/2)nj, for all Bi C {0, 1}"' with ||A|| > 2''^, 
for all B2 C {0, 1}"' with H^all > 2''\ and for aU a G {0, 1}™% 

\\Er\a)nBi x ^all < — ll^i x B2II. 

We take Zi = Ei{xi,yi). 

Finally z = Z1Z2 . . . zi . . .. 
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It is obvious that the above procedure is a truth-table reduction (i.e., it halts on all inputs). 
In what follows we will assume that the two input sequences x and y have randomness rate 
r and our goal is to show that the output z has randomness rate (1 — 5) for any (5 > 0. 

Lemma 4.11 For any e > 0, for all i sufficiently large, K{zi \ > (1 — e) • rui. 

Proof. Suppose K{zi \ xt-iyi-i) < (1 — e) • mj. 

Let A = {ze {0, 1}™^ I K{z \ Xi_m_i) < (1 - e) • mj. We have \\A\\ < 2(i-^)'^\ 
Let ti,t2, Bi, B2 be defined as follows: 

• ti = K{xi I 

Since K{xi \ > aui, and taking into account Lemma [4.6[ it follows that ti > 

aui — 0{i) > {a/2)ni, for all i sufficiently large. 

• t2 = K{yi I Xi^iyi^i). 

By the same argument as above, ^2 > (o'/2)ni. 

• Bi = {x£ {0, 1}"' I Kix I Xi-m-i) < h}. 

• B2 = {ye {0, 1}"' I K{y \ Xi^im-i) < ^2}. 

We have \\Bi\\ < 2*^+^ Take B[ such that \\B[\\ = 2*1+^ and Bi C B[. 
We have \\B2\\ < 2*2+1. ^pj^j^g ^/ ^^^^i that ||5^|| = 2*2+1 ^ 

The bounds on ti and t2 imply that B[ and B2 are large enough for Ei to satisfy the 
regularity property on them. In other words, for any a £ {0, 1}™-* , 

\\Er\a)nB[xB',\\<^\\B[xB',\\. 

So, 

\\Er\A)nBi X S2II < ll^ri(A) nSj x B'^W 

= Ea^A\\ErHa)nB[xB',\\ 

< 2(i-^)'"'2^||s; X B'^W 

2*l+*2— <:"^i+3 

There is an algorithm that, given {xi,X2, ■ ■ ■ ,Xi-i), {yi,y2, ■ ■ ■ ,yi-i), (1 — e)mj, ti and t2, 
enters an infinite loop during which it enumerates the elements of the set E^^(A) D Bi x S2. 
Therefore, the Kolmogorov complexity of any element of E^^{A) D Bi x B2 is bounded by 
its rank in some fixed enumeration of this set, the binary encoding of the input (including 
the information needed to separate the different components), plus a constant number of bits 
describing the enumeration procedure. 

Formally, for every {u,v) G ^-^(^)n-Bi X B2, 

K{uv I Xi-iyi-i) <ti+t2- emi + 2(log(l - e)mi + log ti + log t2) + 0(1) = h + t2 - n{i^). 

We took into account that rui = i"^, logti = 0{i), and logt2 = 0{i). In particular, 

K{xiyi I Xi-iyi-i) <ti + t2- O(i^). 

On the other hand, by Lemma 14.7^ 

K{xiyi I Xi-iyi^i) > K{xi \ Xi^m-i) + K{yi \ Xi^m-i) - 0{i) 
= ti + t2-0{i). 

The last two inequations are in conflict, and thus we have reached a contradiction. | 
The following lemma concludes the proof of the main result. 
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Lemma 4.12 For any 5 > 0, the sequence z obtained by concatenating in order zi, Z2, ■ ■ ■, has 
randomness rate at least 1 — 5. 

Proof. Take e = (5/4. 

By Lemma [4.111 K{zi \ > (1 — e) • rrij, for all i sufficiently large. 

This implies K{zi \ zi . . . > (1 — e)mj — 0(1) > (1 — 2e)mj (because each Zj can be 
effectively computed from Xj and yj). 

By induction, it can be shown that K{zi . . . Zi) > (1 — 3e)(mi + . . . + For the inductive 
step, we have 

K{ziZ2 ■■■Zi) > K{zi . . . Zi-i) + K{zi I zi . . . Zi^i) - 0(log(mi + . . . + mi_i) + log(mj)) 

> (1 - 3e)(mi + . . . + mi_i) + (1 - 2e)mj - 0(log(mi + . . . + rm)) 

> (l-3e)(mi + ... + mi). 

Now consider some z' which is between zi . . . zi-i and zi . . . Zj, i.e., for some strings u and 
V, z' = zi . . . Zi-iu and zi . . . Zi = z'v. Suppose K{z') < (1 — 4e)|2;'|. 
Then zi . . . zt-i can be reconstructed from: 

(a) the descriptor of z' , which takes (1 — 4e)|z'| < (1 — 4e)(mi + . . . + rrij) bits, 

(b) the string u which takes — l^i . . . Zj-i] < mj bits 

(c) O(logmi) bits needed for separating u from the descriptor of z' and for describing the 
reconstruction procedure. 

This implies that 

K{zi . . . Zi^i) < (1 - 4e)(mi + . . . + rrii) + rrii + O(logmj) 

= (1 - 4e)(mi + . . . + (2 - 4e)mj + O(logmj) 

= (1 - 4e + (2 - 4e);J-^) • (mi + . . . + + 0(log m,) 

< (1 - 3e)(mi + . . . + mj_i). 

(The last inequality holds if ^ goes to 0, which is true for mj = i"^.) This is a 

contradiction. 

Thus we have proved that for every n sufficiently large, K(z{l : n)) > (1 — 6)n. | 

The main result can be stated in terms of constructive Hausdorff dimension, a notion 
introduced in measure theory. The constructive Hausdorff dimension of a sequence x G {0, 1}°° 
turns out to be equal to liminf ^(^(^-"^^^ (gee |May02 Rya84 IStaOSj ). 

Corollary 4.13 For any r > 0, there is a truth-table reduction f such that if x £ {0,1}°° 
and y S {0, 1}°° are independent and have constructive Hausdorff dimension at least t, then 
f{x,y) has Hausdorff dimension 1. Moreover, f is uniform in the parameter r. 

We next observe that Theorem 11.11 can be strengthened by relaxing the requirement regarding 
the independence of the two input sequences. For a function g : N ^ M"^, we say that two 
sequences x £ {0, 1}°° and y S {0, 1}°° have dependency g, if for all natural numbers n and 
m, 

K{x{l : n)) + K{y{l : m)) - K{x{l : n)y(l : m)) < 0{g{n) + g{m)). 

In Theorem ll.il the assumption is that the two input sequences have dependency g{n) = logn. 
Using essentially the same proof as the one that demonstrated Theorem 11.11 one can obtain 
the following result. 

Theorem 4.14 For any r > 0, there exist < a < 1 and a truth-table reduction f : 
{0, 1}°° X {0, 1}°° ^ {0, 1}°° such that if x e {0, 1}°° and y G {0, 1}°° have dependency 
and randomness rate r, then f{x,y) has randomness rate 1 — S, for any positive 5. Moreover, 
f is uniform in the parameter r. 
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In Theorem II. II it is required that the initial segments of x and y have Kolmogorov complexity 
at least r -n, for a positive constant r. We do not know if it is possible to obtain a similar result 
for sequences with lower Kolmogorov complexity. However, using the same proof technique, 
it can be shown that if x and y have their initial segments with Kolmogorov complexity 
only J7(logn), then one can produce an infinite sequence z that has very high Kolmogorov 
complexity for infinitely many of its prefixes. 

Theorem 4.15 For any 5 > there exist a constant C and a truth-table reduction f : 
{0, 1}°° X {0, 1}°° — > {0, 1}°° with the following property: 

If the input sequences x and y are independent and satisfy K{x(l : n)) > C ■ logn and 
K{y(l : n)) > C ■ logn, for every n, then the output z = f{x,y) satisfies K{z{\ : n)) > 
(1 — 5) • n, for infinitely many n. Furthermore, there is an infinite computable set S, such that 
K{z{l : n)) > {1 — 5) ■ n, for every n £ S. 
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